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Summary  of  Results 


In  this  section  we  will  present  a  brief  summary  of  our  research  results 
published  after  October  1,  1981.  In  the  following  section  we  will  give  a 
brief  overview  of  our  research  in  progress.  Then  in  the  final  section  we  will 
present  a  list  of  publications  supported  by  the  Grant  AF0SR-81-0047  since 
October  1,  1981.  The  references  in  the  present  section  are  keyed  to  the  publi¬ 
cation  list  in  the  last  section. 

Much  of  our  work  during  the  previous  grant  year  was  concerned  with  quantiza¬ 
tion  theory.  Quantization  forms  the  heart  of  analog  to  digital  conversion  and  it 

is  a  key  element  in  virtually  all  of  digital  signal  processing.  An  N-level 

k  k 

k-dimensional  vector  quantizer  is  a  mapping  Q:  F  F  which  assigns  the  input 
vector  x  to  an  output  vector  Q(x)  chosen  from  a  finite  set  of  N  distinct  vectors 

L 

(y..:y..  e  F  ,  1»1,2,...,N).  Generally,  the  quantizer  input  is  modelled  as  a 
random  vector  X  described  by  a  k-variate  distribution  F.  A  measure  of  quantizer 
performance  is  the  distortion  function 

D(Q,F)  *  /c[x  -  Q(x)]dF(x) , 

where  C(-)  Is  an  appropriately  chosen  cost  function,  for  example,  C(x)  *  ||x||  , 
where  [|  *||  denotes  the  Euclidean  norm  on  F^,  results  in  the  mean  squared  error. 
Thus  the  quantizing  operation  entails  a  coarsening  or  reduction  of  available 
Information,  in  the  sense  that  all  points  in  the  set  R.  ■'  {x:  Q{x)  *  y^)  are 
Identified  together. 

Although  quantization  theory  has  been  a  subject  of  interest  for  many  years, 
much  remains  to  be  done  in  this  area.  The  difficulty  in  this  area  is  compounded 
by  the  fact  that  some  of  the  commonly  held  beliefs  are  actually  misconceptions. 
For  example.  It  has  often  been  assumed  in  the  literature  that  in  the  scalar 
case  a  minimum  mean  squared  error  optimum  quantizer  for  a  symmetrically  distri¬ 
buted  random  variable  will  be  symmetric  (i.e.  the  quantization  levels  will  be 
symmetrically  distributed  about  the  origin).  ^In  [10]  we  gave  the  following 
result  for  scalar  quantizers. 

Theorem:  Let  the  cost  function  be  given  by  C(x)  *  [x|r  for  some  positive 
r.  Let  N  be  a  positive  even  Integer  and  let  fQ  be  a  given  symmetric  density 
function  with  finite  r-th  absolute  moment.  For  any  given  c,  0  <  e  <  1,  there 
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exists  a  symmetric  density  function  h  such  that  for  the  mixture  density 
(l-e)fg  +  eh,  no  optimal  N-level  scalar  quantizer  is  symmetric. 

An  open  problem  in  the  subject  of  quantization  theory  had  been  the  exis¬ 
tence  of  optimal  quantizers  (i.e.  could  the  infimum  over  all  N-level  quantizers 
be  achieved?).  In  [15]  we  showed  that  if  C(x)  =  Cq(||x||),  where  CqU)  is  non- 
neqative,  real  valued,  nondecreasinq  for  t  >  0,  and  lower  semi -continuous,  then 
an  optimum  N-level  quantizer  exists  for  any  positive  integer  N.  In  [10]  we 
extended  this  result  to  the  case  where  if  Cq( t )  is  non-negative  and  non-decreasing 
for  t  >  0,  then  a  necessary  and  sufficient  condition  that,  for  any  N  and  any 
distribution  function,  a  minimum  distortion  N-level  quantizer  exists,  is  that 
CqU)  be  lower  semi -continuous.  Also  in  [2]  we  showed  that  if  the  distribution 
of  the  random  vector  X  is  continuous,  then  for  any  cost  function  C(x)  ■  Cq(||x||), 
where  Cg(t)  is  nonnegative  and  nondecreasing  for  t  >  0,  a  globally  minimum  dis¬ 
tortion  N-level  quantizer  exists  for  every  N.  For  example,  the  well-known  Cantor 
distribution  is  continuous  (although  singular  with  respect  to  Lebesgue  measure); 
nevertheless,  any  such  cost  function  yields  a  distortion  that  can  be  minimized. 

In  [1]  and  [17]  we  investigated  some  convergence  properties  of  sequences  of 
quantizers  (and  as  a  side  result  we  established  existence  of  optimal  quantizers 
via  convergence  arguments).  Specifically,  we  were  concerned  with  the  following. 

L 

Suppose  that  a  sequence  { Fn>  of  probability  distribution  functions  on  F  con¬ 
verges  weakly  to  a  distribution  function  F.  When  does  the  sequence  of  optimal 
quantizers  (Qn)  for  the  Fn's  converge  to  an  optimal  quantizer  0  for  F?  When 
does  the  sequence  of  distortions  D(Qn,Fn)  converge  to  D(Q,F)?  In  [1]  and  [17] 
we  discussed  conditions  under  which  this  convergence  can  take  place.  In  recent 
years  much  work  has  been  done  to  investigate  design  algorithms  for  optimal  quan¬ 
tization.  A  popular  approach  is  to  use  numerical  techniques  to  successively 
improve  on  a  sequence  of  sub-optimal  designs  using  conditions  which  are  necessary 
(but  not  sufficient)  for  a  stationary  point.  Linde,  Buzo,  and  Gray  ("An  Algorithm 
for  Vector  Quantizer  Design,"  IEEE  Transactions  on  Communications,  Vol.  COM-28, 
pp.  84-95,  January  1980)  recently  presented  an  algorithm  (essentially  a  fixed 
point  algorithm)  for  quantizer  design  that  handles  arbitrary  multi-dimensional 
distributions  and  very  general  distortion  measures,  and  they  conjectured  that 
this  algorithm  would  converge  to  a  global  minimum  but  were  unable  to  prove  it 
rigorously.  In  [1]  and  [17]  we  proved  convergence  of  this  algorithm  for  quantizer 
design. 

In  [6]  we  considered  asymptotic  properties  of  the  r-th  power  distortion. 
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l.e.  £(||  X-Q(X)||r } ,  associated  with  quantized  k-dimensional  random  variables. 
Subject  only  to  a  moment  condition, we  showed  In  [6]  that  the  Inflmum  over  all 
N-level  quantizers  of  the  quantity  Nr^  times  the  r-th  power  distortion  measure 
converges  to  a  finite  constant  as  N  -*■«.  This  result  might  be  utilized  in  the 
following  way.  Let  QN  denote  an  optimal  N-level  quantizer.  How  big  should  N 
be  so  that  the  r-th  power  distortion  Is  acceptably  small?  Since 
Nr^k  £  {||  X-QN (X )  jj  r>  -*■  k,  if  N  Is  sufficiently  large  we  might  invoke  the  approx¬ 
imation  E  { ||  X-Qm(X)|j  r}  *  k  N-r^.  Notice  that  our  work  in  [6],  where  we  con¬ 
sidered  an  example  involving  the  Cantor  distribution,  stands  in  marked  contrast 
to  the  following  statement,  taken  from  Yamada,  Tazaki,  and  Gray  ("Asymptotic 
Performance  of  Block  Quantizers  with  Difference  Distortion  Measures,"  IEEE 
Transactions  on  Information  Theory,  Vol.  IT-26,  pp.  6-14,  January  1980): 

"The  initial  fundamental  assumption  in  all  studies  of  asymptotic  quantization 
is  that  the  probability  density  p(x)  is  sufficiently  'smooth'  to  ensure  that 
p(x)  is  effectively  constant  over  small  bounded  sets." 

In  the  context  of  scalar  quantizers,  any  quantizer  can  be  realized  via 
the  companding  approach;  that  is,  a  strictly  increasing  nonlinearity  (the  com¬ 
pressor)  mapping  the  reals  into  [0,1]  followed  by  a  uniform  quantizer  on  [0,1] 
followed  in  turn  by  the  Inverse  of  the  first  nonlinearity  (the  expander).  In 
[16]  we  investigated  the  situation  where  the  output  of  the  uniform  quantizer  is 
transmitted  through  a  noisy  channel  and  the  output  of  this  channel  is  then  put 
through  the  expander.  The  compressor  was  designed  taking  into  account  both 
the  effects  of  quantization  errors  and  channel  errors.  Then  in  [3]  we  considered 
a  slightly  different  approach  to  companding,  where  the  compander  was  constrained 
to  be  piecewise  linear.  We  designed  the  (asymptotically)  optimal  compander  sub¬ 
ject  to  the  constraint  that  It  be  piecewise  linear,  and  we  investigated  several 
properties  of  such  piecewise  linear  companders. 

In  [11]  we  Investigated  an  existing  method  (E.J.  Delp  and  O.R.  Mitchell, 
"Image  Compression  Using  Block  Truncation  Coding,”  IEEE  Transactions  on  Commun¬ 
ications,  Vol.  COM-27,  pp.  1335-1342,  September  1979)  for  image  compression 
known  as  block  truncation  coding.  The  basic  block  truncation  coding  approach 
employs  a  one  bit  (i.e.  two  level)  nonparametri c  quantizer  whose  output  levels 
are  obtained  through  matching  the  first  two  sample  moments  of  the  data  before 
and  after  quantization.  In  [11]  we  generalized  this  basic  block  truncation 
coding  approach  by  employing  one  bit  nonparametrlc  quantizers  which  preserve 
higher  order  moments.  This  generalization  offered  the  potential  for  Improved 
performance.  We  illustrated  by  way  of  example  that  such  Improvement  was 
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indeed  possible  under  the  criteria  of  mean  absolute  error  and/or  mean  squared 
error. 

The  matched  filter  has  been  of  practical  interest  for  some  time.  This 
filter,  which  maximizes  the  output  signal  to  noise  power  ratio,  requires  know¬ 
ledge  of  the  signal  input  for  its  design.  In  many  cases  it  is  reasonable  to 
expect  that  the  signal  will  be  known  at  various  discrete  instants,  thus  admitting 
the  design  of  the  discrete  time  filter.  Unfortunately,  while  it  is  reasonable 
to  expect  that  the  input  signal  will  be  known  at  discrete  instants,  it  is  another 
matter  to  assume  that  it  will  be  known  exactly  as  a  closed  form  analytical 
expression  over,  for  example,  an  interval  of  time,  wriich  would  be  necessary  for 
the  design  of  a  matched  filter  in  continuous  time.  Design  of  the  continuous  time 
filter  is  thus  inhibited  by  such  inexact  knowledge  of  the  signal  input. 

While  the  signal  may  be  incompletely  known,  it  is  reasonable  to  expect  that 
in  many  cases  it  could  be  modeled  as  bandlimited.  If  we  furthermore  assume  the 
signal  is  known  at  a  fixed  number  of  instants,  we  might  hope  that  a  continuous 
time  filter  could  be  designed  which  is  insensitive  to  the  remaining  Inexactness 
in  our  knowledge  of  the  signal.  In  [2]  and  [18]  we  presented  an  approach  toward 
inproving  the  signal  to  noise  power  ratio  of  a  discrete  time  matched  filter  by 
employing  a  continuous  time  filter.  This  approach  exploited  the  bandlimited 
nature  of  the  unknown  continuous  time  signal,  and  resulted  in  a  filter  which  is 
robust  to  this  inexact  knowledge. 

The  relative  efficiency  between  two  detectors  Is  a  measure  of  the  amount 
of  data  one  detector  requires,  relative  to  the  other  detector,  to  attain  a  pre¬ 
scribed  level  of  performance.  Often  the  asymptotic  relative  efficiency  (ARE) 
is  employed  as  a  criterion  for  comparison  of  detectors.  (In  the  context  of 
hypothesis  testing,  this  is  known  in  the  statistical  literature  as  the  Pitman 
efficiency.)  The  ARE  is  generally  held  to  be  appropriate  in  the  case  of  large 
sample  size  and  small  signal  strength.  Moreover,  the  employment  of  the  ARE 
generally  yields  mathematically  tractable  results,  due  largely  to  the  applica¬ 
bility  of  central  limit  theorems.  While  it  is  not  our  intention  to  disqualify 
the  ARE  as  a  measure  of  detector  performance,  our  investigations  do  question  its 
validity  as  a  universal  measure  of  detector  performance.  In  [9]  we  investigated 
the  behavior  of  two  pairs  of  popular  detection  systems,  and  we  found  that  in 
some  cases  the  ARE  can  be  a  poor  indicator  of  finite  sample  size  detection 
performance  even  for  some  fairly  large  sample  sizes.  Thus,  in  some  cases  if  the 
relative  efficiency  is  assumed  to  converge  quickly,  the  "wrong"  detector  might 
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be  chosen  for  a  particular  application.  This  behavior  was  seen  to  be  particularly 
true  for  heavy-tailed  noise  models  such  as  Laplace  noise. 

Kassam  and  Thomas  ("A  Class  of  Nonparametric  Detectors  for  Dependent  Input 
Data,"  IEEE  Transactions  on  Information  Theory,  Vol.  IT-21,  pp.  431-437,  July  1975) 
considered  detecting  a  constant  signal  in  m-dependent  noise.  This  scheme  consisted 
of  summing  the  first  n  samples,  skipping  (i.e.  throwing  away)  the  next  m,  summing 
the  next  n,  skipping  the  next  m,  etc.  They  then  applied  the  classical  sign 
detector  to  the  sequence  of  sums,  and  they  concluded  that,  asymptotically,  n 
should  be  chosen  as  large  as  possible  to  maximize  performance.  In  [8]  we  inves¬ 
tigated  the  performance  of  this  nonparametric  scheme  in  both  the  asymptotic  case 
and  the  case  of  a  finite  number  of  samples.  We  showed  that  the  design  of  the 
detector  with  a  finite  amount  of  data  can  be  radically  different  from  the  design 
based  on  the  asymptotic  limit. 

In  several  applications  of  discrete  time  signal  processing,  a  nonlinear 
scheme  called  median  filtering  has  achieved  some  very  interesting  results.  The 
implementation  of  a  median  filter  requires  a  very  simple  nonlinear  operation. 
Consider  a  fixed  nonnegative  integer  N.  At  a  given  time  instant,  the  output  of 
the  median  filter  is  the  empirical  median  of  the  samples  lying  within  a  window 
centered  at  the  given  time  instant  and  spanning  2N+1  adjacent  samples.  Tukey 
is  generally  credited  with  introducing  the  concept  of  median  filters,  and  he  did 
much  of  the  pioneering  work  in  this  area.  Although  the  concept  of  median  filters 
was  first  Introduced  in  the  statistics  literature,  it  soon  found  applications 
In  the  area  of  signal  processing.  Loosely  speaking,  linear  filtering  tends  to 
smooth  out  abrupt  changes  in  a  sequence  of  data,  but  a  median  filter  is  capable 
of  preserving  sharp  changes  in  data.  In  [4]  we  presented  the  first  rigorous 
analysis  of  the  properties  of  median  filters.  We  precisely  characterized  which 
signals  could  pass  through  a  median  filter  unchanged,  we  characterized  the  conceot 
of  a  root  of  a  median  filter,  and  we  showed  that  with  repeated  median  filtering 
we  would  eventually  produce  a  root  signal  of  the  median  filter.  In  [5]  we  con¬ 
sidered  the  effect  of  a  median  filter  upon  the  spectral  density  of  a  homogeneous 
Markov  chain.  In  [12]  we  analyzed  the  effect  of  a  median  filter  on  an  input 
which  consisted  of  the  sum  of  a  known  constant  sional  and  independent  Identically 
distributed  noise.  Specifically,  we  studied  the  probability  that  the  difference 
between  the  output  of  the  filter  and  the  constant  signal  is  close  to  zero.  This 
probability  can  be  used  to  determine  the  performance  of  the  filter  on  a  per-sample 
basis.  Some  examples  were  presented  to  illustrate  the  results. 
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A  model  that  frequently  arises  in  engineering  applications  is  given  by 
the  differential  equation 

X(t)  =  A(t)X(t) 

L 

where  t  ^  0,  X{t)  e  R  ,  X(0)  =  Xq,  and  A(t)  is  a  k>k  random  matrix.  For 
example,  by  taking 

A(t)  =  G  +  N(t)H, 

where  N(t)  is  a  random  process  and  H  and  G  are  k*k  matrices,  we  arrive  at  a 
bilinear  system.  (Bilinear  systems  and  their  relations  to  more  general  non¬ 
linear  systems,  e.g.  Volterra  systems,  have  been  investigated  by  other  researchers.) 
Systems  of  this  general  form  also  arise  in  modelling  faulty  systems  (the  random 
changing  of  A(t)  is  used  to  represent  the  effects  of  faulty  devices  in  a  system). 
Numerous  investigations  have  been  concerned  with  the  establishment  of  conditions 
guaranteeing  that  as  t  -*■  »,  ||  X(t)||  -*•  0  in  some  probabilistic  sense,  where  ||-|| 
is  some  norm  on  R  .  We  have  recently  established  conditions  [13]  guaranteeing 
that  ||  X(t)||  -*■  0  with  probability  one,  and  these  conditions  are  more  general  than 
others  dealing  with  this  form  of  problem.  We  do  not  require  the  components  of 
A(t)  to  be  ergodic,  and  we  permit  them  to  even  possess  "badly  behaved"  sample 
functions;  for  example,  sample  functions  possessing  a  discontinuity  of  the 
second  kind  at  every  point  and  that  are  unbounded  in  any  interval  of  positive 
length.  Our  conditions  are  expressed  in  terms  of  second  moment  properties 
of  the  components  of  A(t). 

In  [7]  we  considered  the  prediction  of  a  time  series,  where  our  predictor 
was  constrained  to  have  the  form  of  a  zero  memory  nonlinearity  followed  by  a 
linear  filter.  Systems  such  as  this  can  always  perform  at  least  as  well  as 
linear  systems,  and  in  many  cases  significantly  outperform  linear  systems. 

Minimum  shift  keying  (MSK)  Is  a  popular  digital  modulation  technique  which 
is  known  for  Its  property  of  bandwidth  conservation.  Boutin  and  Morissette  ("Do 
All  MSK-type  Signalling  Waveforms  Have  Wider  Spectra  than  those  for  PSK?,"  IEEE 
Transactions  on  Communications,  Vol.  COM-29,  pp.  1071-1072,  July  1981)  made 
an  erroneous  attempt  to  provide  a  bound  on  the  width  of  the  center  spectral 
lobe  of  MSK-type  signaling  waveforms.  In  [14]  we  provided  a  correct  development 
of  this  property  which  yielded  a  bound  on  the  width  of  the  center  spectral  lobe 
of  MSK-type  signaling  waveforms. 
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Research  in  Progress 

During  the  present  grant  year  our  research  is  concentrating  upon  several 
problem  areas.  One  problem  area  which  we  are  vigorously  pursuing  is  quantiza¬ 
tion  theory.  Our  earlier  work  had  dealt  with  difference-based  distortion 
measures,  that  is,  distortions  such  as 

/  C[x-Q(x)]dF(x). 

We  are  presently  extending  this  to  nondifference-based  distortion  measures  of 
the  form 

/d[x,Q(x)]dF(x). 

It  is  known  that  difference-based  distortions  are  not  always  satisfactory, 
especially  when  held  up  against  subjective  evaluations.  Recently,  more  com¬ 
plicated  cost  functions  have  been  investigated  in  an  effort  to  obtain  more 
meaningful  distortion  measures.  We  plan  to  investigate  existence  of  optimal 
quantizers  and  also  convergence  properties  of  sequences  of  quantizers.  Also, 
we  plan  to  extend  the  space  being  quantized  from  IR  to  metric  spaces.  Quanti¬ 
zation  of  metric  spaces  may  be  potentially  useful  as  a  tool  in  extending  theorems 
in  information  theory  and  coding  theory  to  function  spaces. 

Another  problem  area  we  will  be  concerned  with  is  the  theory  of  signal 
detection.  We  plan  to  study  several  questions  concerned  with  approximations 
in  detection  systems;  for  example,  the  digital  implementation  of  an  analog 
scheme  for  detection.  Also,  we  plan  to  study  the  relative  efficiency  (i.e.  with 
a  finite  number  of  samples)  between  pairs  of  detectors  for  time-varying  signals. 

Another  problem  we  are  considering  is  concerned  with  the  effect  of  a  zero 
memory  nonlinearity  on  the  spectrum  of  certain  classes  of  random  processes. 

That  is,  let  X(t)  be  a  stationary  random  process  possessing  a  spectral  repre¬ 
sentation,  and  let  g:  F  +  F  be  a  Borel  measurable  function  such  that 
2 

E((g[X(t)])  }  <  »  .  Then  what  can  we  say  about  the  spectrum  of  g[X(t)]  as 
compared  to  that  of  X( t ) ?  When  is  one  more  spread  out  than  the  other?  This 
problem  thus  relates  to  the  bandwidth  of  nonlinearly  distorted  random  processes. 
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